Pseudo-Entropy Based Pruning Algorithm for Feed forward Neural Networks

Design of artificial neural networks is an important and practical task:"how to choose the adequate size of neural architecture for a given application". One popular method to overcome this problem is to start with an oversized structure and then prune it to obtain simpler network with a good generalization performance. This paper presents a pruning algorithm based on pseudo-entropy of hidden neurons. The pruning is performed by iteratively training of network to a certain performance criterion and then removing the hidden neuron with individual pseudo-entropy greater than a preselected threshold which is slightly higher than the average value of network's pseudo-entropy until no one can further be removed. This approach is validated with an academic example and it is tested on induction motor modeling problem. Compared with two modified versions of Optimal Brain Surgeon (OBS) algorithm, the developed method gives interesting results with an easier computation tasks.